CamelCamelCamel API Design Decisions

This lesson will discuss design decisions paramount to creating an effective C3 service. Mainly, we’ll decide on a suitable architectural style, data format, and HTTP version for an effective C3 service.

Design overview#

Let’s start by understanding the architecture and workflow of the C3 service before diving into a discussion of design decisions because C3’s design architecture and workflow will enable us to make suitable decisions. Notice the synchronous communication of the C3 service with Amazon, a distinct service provider. The C3 API uses search, pub-sub, and product services. These services interact with Amazon’s services to provide data to potential customers. The following illustration demonstrates a high-level architecture of the workings of a C3 service:

The following table gives an overview of each of the components involved in a C3 service:

Workflow of C3#

The C3 service works synchronously with the Amazon service. Let’s discuss the workflow of each service according to the functional requirements. Requests to C3’s API gateway can interact with any of the subservices of C3 according to the required functionality, as described below:

• Search product: If it's a search query from a user, the API gateway sends it to the search service of the C3. The C3’s search service further collaborates with Amazon’s search service. Interacting with Amazon’s search service allows C3 to produce the most accurate results for the client. After receiving a query from C3, Amazon’s search service processes it, generates results, and delivers them back to C3’s search service. It is now C3’s responsibility to process data and send a response back to the client.

• Adding a product to price watch: If clients want to receive a price drop alert of a product that was searched for earlier using a search service or that was suggested by the C3 service, they need to subscribe to the product with a desired price. Such a request is forwarded to the pub-sub service from the API gateway. It keeps products as topics, and whenever a price drop (event) occurs, it notifies the client (subscriber). To evaluate price drops, the pub-sub service acquires information from the product service of C3. Similarly, clients can get a list of the products from the pub-sub service that were added to their watch list.

• Price timeline: The price timeline is acquired using the product service of C3. The product service communicates with Amazon’s product advertising service to get up-to-date information about the products. It uses the SQS service of Amazon to send periodic requests to Amazon’s product advertising service.
  The data received in response is compared with local data of the C3’s product service. The C3’s database gets updated in case of updated results from Amazon’s product service. It also informs the pub-sub service about the prices and details of the products added to the watch list. The workflow of SQS and product advertising API is discussed below:

  • SQS service: The PA-API of Amazon allows a limited number of requests per user since there are otherwise millions of interactions daily. SQS provides queuing functionality to perform tasks (processing requests in our case) periodically. C3 prepares several requests and pushes them to the SQS. The SQS processes each request from the queue. The pattern of each request is that it takes query parameters and asks Amazon’s product advertising service about the data based on the queries. It delivers the response to C3’s product service, as shown in the illustration below.

  • Product advertising service: This service of Amazon provides up-to-date information about millions of products. Requests from SQS are forwarded to the product advertising service, which processes the request, prepares a response, and sends it back to the requesting service.

Note: How periodic requests work is that the PA-API defines rate limiting to address millions of users, so the queue is filled with requests to be sent periodically according to the allowed number of requests in a specified period.

Design considerations#

This section discusses the design considerations of architectural styles, data formats, and HTTP protocols.

Architectural styles#

We have four layers of communication between a client and synchronous services. These layers are listed below:

• Client to API gateway

• API gateway to C3 service

• C3 service to API gateway

• API gateway to Amazon service

Different operations are performed through the request-response architecture between each layer of communication. All the operations, like searching, reading price history, and sending a subscription request, are simple CRUD operations and resource-oriented. Therefore, we can use the REST API architectural style for communication due to its simple and standardized way of managing resources. Moreover, from the workflow, it is clear that there is no interaction between multiple resources simultaneously for any operation. Though we have synchronous communication between resources, we do not fetch data from multiple resources; therefore, graphQL is not a viable option. Moreover, C3 service depends on Amazon due to the synchronous communication. REST is also preferred due to its compatibility with Amazon.

Communication protocols#

The choice of a suitable HTTP protocol is a bit more complex in this case. That is because we have to consider both the communication between the client and C3 service and the C3 service and Amazon’s API. The operations are in the simple request-response format and do not need multiple responses (continuous stream of data) for a single request. We can use a persistent connection of HTTP/1.1 to perform simple CRUD operations while interacting with the C3 service. Most applications use the HTTP/1.1 protocol for simple interaction with APIs due to its compatible infrastructure—and so does Amazon. Therefore, HTTP/1.1 seems suitable for the C3 service to have compatible communication with the Amazon service.

The pub-sub service, however, uses event-driven protocols such as WebSub to notify customers. So, for event-driven communication in the C3 service, we’ll use the WebSub protocol.

Although we can have the advantage of low latency by using HTTP/2.0 protocol to fetch a large amount of data, it incurs extra complexity to implement.

Data formats#

The choice of data format is relatively straightforward in this case. Our service depends on Amazon, which uses JSON as its data format. We may consider conversion because data format is an important element in the API design, but JSON also seems like a viable option for our API due to readability and compactness.

Summary#

The design decisions are summarized in the following table: